dnsmasq
and
iptables
to be able to run and communicate different virtualization systems
and containers on laptops running Debian GNU/Linux.
I ve used different variations of this setup for a long time with
VirtualBox and KVM for
the Virtual Machines and Linux-VServer,
OpenVZ, LXC and lately
Docker or Podman for the
Containers.
systemd
and network-manager
to configure the
WiFi and Ethernet interfaces, but for the bridge I use bridge-utils
with
ifupdown
(as I said this setup is old, I guess ifupdow2
and ifupdown-ng
will work too).
To start and stop the DNS
and DHCP
services and add NAT rules when the
bridge is brought up or down I execute a script that uses:
ip
from iproute2
to get the network information,dnsmasq
to provide the DNS
and DHCP
services (currently only the
dnsmasq-base
package is needed and it is recommended by network-manager
,
so it is probably installed),iptables
to configure NAT (for now docker
kind of forces me to keep using
iptables
, but at some point I d like to move to nftables
).sudo apt install bridge-utils dnsmasq-base ifupdown iproute2 iptables
ifupdow
is available on the file
/etc/network/interfaces.d/vmbr0
:
# Virtual servers NAT Bridge
auto vmbr0
iface vmbr0 inet static
address 10.0.4.1
network 10.0.4.0
netmask 255.255.255.0
broadcast 10.0.4.255
bridge_ports none
bridge_maxwait 0
up /usr/local/sbin/vmbridge $ IFACE start nat
pre-down /usr/local/sbin/vmbridge $ IFACE stop nat
ifupdown
make sure that /etc/network/interfaces
contains the line:
source /etc/network/interfaces.d/*
/etc/network/interfaces
directly, if you prefer.10.0.4.1
and
assumes that the machines connected to it will use the 10.0.4.0/24
network;
you can change the network address if you want, as long as you use a private
range and it does not collide with networks used in your Virtual Machines all
should be OK.
The vmbridge
script is used to start the dnsmasq
server and setup the NAT
rules when the interface is brought up and remove the firewall rules and stop
the dnsmasq
server when it is brought down.vmbridge
scriptThe vmbridge
script launches an instance of dnsmasq
that binds to the
bridge interface (vmbr0
in our case) that is used as DNS and DHCP server.
The DNS server reads the /etc/hosts
file to publish local DNS names and
forwards all the other requests to the the dnsmasq
server launched by
NetworkManager
that is listening on the loopback interface.
As this server already does catching we disable it for our server, with the
added advantage that, if we change networks, new requests go to the new
resolvers because the DNS server handled by NetworkManager
gets restarted and
flushes its cache (this is useful if we connect to a new network that has
internal DNS servers that are configured to do split DNS for internal services;
if we use this model all requests get the internal address as soon as the DNS
server is queried again).
The DHCP server is configured to provide IPs to unknown hosts for a sub range
of the addresses on the bridge network and use fixed IPs if the /etc/ethers
file has a MAC with a matching hostname on the /etc/hosts
file.
To make things work with old DHCP clients the script also adds checksums to
the DHCP packets using iptables
(when the interface is not linked to a
physical device the kernel does not add checksums, but we can fix it adding a
rule on the mangle
table).
If we want external connectivity we can pass the nat
argument and then the
script creates a MASQUERADE
rule for the bridge network and enables IP
forwarding.
The script source code is the following:
/etc/NetworkManager/NetworkManager.conf
file has the following
contents:
[main]
plugins=ifupdown,keyfile
[ifupdown]
managed=false
ifupdown
alone and, by
default, will send the connection DNS configuration to systemd-resolved
if it
is installed.
As we want to use dnsmasq
for DNS resolution, but we don t want
NetworkManager
to modify our /etc/resolv.conf
we are going to add the
following file (/etc/NetworkManager/conf.d/dnsmasq.conf
) to our system:
and restart the NetworkManager
service:
$ sudo systemctl restart NetworkManager.service
NetworkManager
will start a dnsmasq
service that queries
the servers provided by the DHCP servers we connect to on 127.0.0.1:53
but
will not touch our /etc/resolv.conf
file.
systemd-resolved
If we start using our own name server but our system has systemd-resolved
installed we will no longer need or use the DNS stub; programs using it will
use our dnsmasq
server directly now, but we keep running systemd-resolved
for the host programs that use its native api or access it through
/etc/nsswitch.conf
(when libnss-resolve
is installed).
To disable the stub we add a /etc/systemd/resolved.conf.d/disable-stub.conf
file to our machine with the following content:
# Disable the DNS Stub Listener, we use our own dnsmasq
[Resolve]
DNSStubListener=no
systemd-resolved
to make sure that the stub is stopped:
$ sudo systemctl restart systemd-resolved.service
/etc/resolv.conf
First we remove the existing /etc/resolv.conf
file (it does not matter if it
is a link or a regular file) and then create a new one that contains at least
the following line (we can add a search
line if is useful for us):
nameserver 10.0.4.1
dnsmasq
server launched when we bring up the
vmbr0
for multiple systems:
/etc/nsswitch.conf
and libnss-resolve
is installed it is queried first,
but the systemd-resolved
uses it as forwarder by default if needed),DHCP
for
network configuration and attach their virtual interfaces to our bridge,/etc/resolv.conf
(if we have entries that use loopback addresses the
containers that don t use the host network tend to fail, as those addresses
inside the running containers are not linked to the loopback device of the
host).$ # Bring interface up
$ sudo ifup vmbr0
$ # Check that it is available
$ ip a ls dev vmbr0
4: vmbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN
group default qlen 1000
link/ether 0a:b8:ef:b8:07:6c brd ff:ff:ff:ff:ff:ff
inet 10.0.4.1/24 brd 10.0.4.255 scope global vmbr0
valid_lft forever preferred_lft forever
$ # View the listening ports used by our dnsmasq servers
$ sudo ss -tulpan grep dnsmasq
udp UNCONN 0 0 127.0.0.1:53 0.0.0.0:* users:(("dnsmasq",pid=1733930,fd=4))
udp UNCONN 0 0 10.0.4.1:53 0.0.0.0:* users:(("dnsmasq",pid=1705267,fd=6))
udp UNCONN 0 0 0.0.0.0%vmbr0:67 0.0.0.0:* users:(("dnsmasq",pid=1705267,fd=4))
tcp LISTEN 0 32 10.0.4.1:53 0.0.0.0:* users:(("dnsmasq",pid=1705267,fd=7))
tcp LISTEN 0 32 127.0.0.1:53 0.0.0.0:* users:(("dnsmasq",pid=1733930,fd=5))
$ # Verify that the DNS server works on the vmbr0 address
$ host www.debian.org 10.0.4.1
Name: 10.0.4.1
Address: 10.0.4.1#53
Aliases:
www.debian.org has address 130.89.148.77
www.debian.org has IPv6 address 2001:67c:2564:a119::77
/etc/hosts
and /etc/ethers
files and reload the dnsmasq
configuration
using the vmbridge
script:
$ sudo /usr/local/sbin/vmbridge vmbr0 reload
dnsmasq
server and it reloads the
files; after that we can refresh the DHCP addresses from the client machines or
start using the new DNS names immediately.Jobs
or CronJobs
).
NodeJS
API with endpoints to upload
files and store them on S3 compatible services that were later accessed via
HTTPS, but the requirements changed and we needed to be able to publish folders
instead of individual files using their original names and apply access
restrictions using our API.
Thinking about our requirements the use of a regular filesystem to keep the
files and folders was a good option, as uploading and serving files is simple.
For the upload I decided to use the sftp protocol, mainly because I already
had an sftp container image based on
mysecureshell prepared; once
we settled on that we added sftp support to the API server and configured it
to upload the files to our server instead of using S3 buckets.
To publish the files we added a nginx container configured
to work as a reverse proxy that uses the
ngx_http_auth_request_module
to validate access to the files (the sub request is configurable, in our
deployment we have configured it to call our API to check if the user can
access a given URL).
Finally we added a third container when we needed to execute some tasks
directly on the filesystem (using kubectl exec
with the existing containers
did not seem a good idea, as that is not supported by CronJobs
objects, for
example).
The solution we found avoiding the NIH Syndrome (i.e. write our own tool) was
to use the webhook tool to provide the
endpoints to call the scripts; for now we have three:
PATH
,hardlink
all the files that are identical on the filesystem,mysecureshell
container can be used to provide an sftp service with
multiple users (although the files are owned by the same UID
and GID
) using
standalone containers (launched with docker
or podman
) or in an
orchestration system like kubernetes, as we are going to do here.
The image is generated using the following Dockerfile
:
ARG ALPINE_VERSION=3.16.2
FROM alpine:$ALPINE_VERSION as builder
LABEL maintainer="Sergio Talens-Oliag <sto@mixinet.net>"
RUN apk update &&\
apk add --no-cache alpine-sdk git musl-dev &&\
git clone https://github.com/sto/mysecureshell.git &&\
cd mysecureshell &&\
./configure --prefix=/usr --sysconfdir=/etc --mandir=/usr/share/man\
--localstatedir=/var --with-shutfile=/var/lib/misc/sftp.shut --with-debug=2 &&\
make all && make install &&\
rm -rf /var/cache/apk/*
FROM alpine:$ALPINE_VERSION
LABEL maintainer="Sergio Talens-Oliag <sto@mixinet.net>"
COPY --from=builder /usr/bin/mysecureshell /usr/bin/mysecureshell
COPY --from=builder /usr/bin/sftp-* /usr/bin/
RUN apk update &&\
apk add --no-cache openssh shadow pwgen &&\
sed -i -e "s ^.*\(AuthorizedKeysFile\).*$ \1 /etc/ssh/auth_keys/%u "\
/etc/ssh/sshd_config &&\
mkdir /etc/ssh/auth_keys &&\
cat /dev/null > /etc/motd &&\
add-shell '/usr/bin/mysecureshell' &&\
rm -rf /var/cache/apk/*
COPY bin/* /usr/local/bin/
COPY etc/sftp_config /etc/ssh/
COPY entrypoint.sh /
EXPOSE 22
VOLUME /sftp
ENTRYPOINT ["/entrypoint.sh"]
CMD ["server"]
/etc/sftp_config
file is used to
configure
the mysecureshell
server to have all the user homes under /sftp/data
, only
allow them to see the files under their home directories as if it were at the
root of the server and close idle connections after 5m
of inactivity:
The entrypoint.sh
script is the one responsible to prepare the container for
the users included on the /secrets/user_pass.txt
file (creates the users with
their HOME
directories under /sftp/data
and a /bin/false
shell and
creates the key files from /secrets/user_keys.txt
if available).
The script expects a couple of environment variables:
SFTP_UID
: UID
used to run the daemon and for all the files, it has to be
different than 0
(all the files managed by this daemon are going to be
owned by the same user and group, even if the remote users are different).SFTP_GID
: GID
used to run the daemon and for all the files, it has to be
different than 0
.SSH_PORT
and SSH_PARAMS
values if present.
It also requires the following files (they can be mounted as secrets in
kubernetes):
/secrets/host_keys.txt
: Text file containing the ssh server keys in mime
format; the file is processed using the reformime
utility (the one included
on busybox) and can be generated using the
gen-host-keys
script included on the container (it uses ssh-keygen
and
makemime
)./secrets/user_pass.txt
: Text file containing lines of the form
username:password_in_clear_text
(only the users included on this file are
available on the sftp
server, in fact in our deployment we use only the
scs
user for everything)./secrets/user_keys.txt
: Text file that contains lines of the form
username:public_ssh_ed25519_or_rsa_key
; the public keys are installed on
the server and can be used to log into the sftp
server if the username
exists on the user_pass.txt
file.entrypoint.sh
script are:
The container also includes a couple of auxiliary scripts, the first one can be
used to generate the host_keys.txt
file as follows:
$ docker run --rm stodh/mysecureshell gen-host-keys > host_keys.txt
.tar
file that contains auth data
for the list of usernames passed to it (the file contains a user_pass.txt
file with random passwords for the users, public and private ssh keys for them
and the user_keys.txt
file that matches the generated keys).
To generate a tar
file for the user scs
we can execute the following:
$ docker run --rm stodh/mysecureshell gen-users-tar scs > /tmp/scs-users.tar
user_pass.txt
file we can do:
$ tar tvf /tmp/scs-users.tar
-rw-r--r-- root/root 21 2022-09-11 15:55 user_pass.txt
-rw-r--r-- root/root 822 2022-09-11 15:55 user_keys.txt
-rw------- root/root 387 2022-09-11 15:55 id_ed25519-scs
-rw-r--r-- root/root 85 2022-09-11 15:55 id_ed25519-scs.pub
-rw------- root/root 3357 2022-09-11 15:55 id_rsa-scs
-rw------- root/root 3243 2022-09-11 15:55 id_rsa-scs.pem
-rw-r--r-- root/root 729 2022-09-11 15:55 id_rsa-scs.pub
$ tar xfO /tmp/scs-users.tar user_pass.txt
scs:20JertRSX2Eaar4x
nginx-scs
container is generated using the following Dockerfile
:
ARG NGINX_VERSION=1.23.1
FROM nginx:$NGINX_VERSION
LABEL maintainer="Sergio Talens-Oliag <sto@mixinet.net>"
RUN rm -f /docker-entrypoint.d/*
COPY docker-entrypoint.d/* /docker-entrypoint.d/
docker-entrypoint.d
scripts from the
standard image and adding a new one that configures the web server as we want
using a couple of environment variables:
AUTH_REQUEST_URI
: URL to use for the auth_request
, if the variable is not
found on the environment auth_request
is not used.HTML_ROOT
: Base directory of the web server, if not passed the default
/usr/share/nginx/html
is used.nginx
image.
The contents of the configuration script are:
As we will see later the idea is to use the /sftp/data
or /sftp/data/scs
folder as the root of the web published by this container and create an
Ingress
object to provide access to it outside of our kubernetes cluster.webhook-scs
container is generated using the following Dockerfile
:
ARG ALPINE_VERSION=3.16.2
ARG GOLANG_VERSION=alpine3.16
FROM golang:$GOLANG_VERSION AS builder
LABEL maintainer="Sergio Talens-Oliag <sto@mixinet.net>"
ENV WEBHOOK_VERSION 2.8.0
ENV WEBHOOK_PR 549
ENV S3FS_VERSION v1.91
WORKDIR /go/src/github.com/adnanh/webhook
RUN apk update &&\
apk add --no-cache -t build-deps curl libc-dev gcc libgcc patch
RUN curl -L --silent -o webhook.tar.gz\
https://github.com/adnanh/webhook/archive/$ WEBHOOK_VERSION .tar.gz &&\
tar xzf webhook.tar.gz --strip 1 &&\
curl -L --silent -o $ WEBHOOK_PR .patch\
https://patch-diff.githubusercontent.com/raw/adnanh/webhook/pull/$ WEBHOOK_PR .patch &&\
patch -p1 < $ WEBHOOK_PR .patch &&\
go get -d && \
go build -o /usr/local/bin/webhook
WORKDIR /src/s3fs-fuse
RUN apk update &&\
apk add ca-certificates build-base alpine-sdk libcurl automake autoconf\
libxml2-dev libressl-dev mailcap fuse-dev curl-dev
RUN curl -L --silent -o s3fs.tar.gz\
https://github.com/s3fs-fuse/s3fs-fuse/archive/refs/tags/$S3FS_VERSION.tar.gz &&\
tar xzf s3fs.tar.gz --strip 1 &&\
./autogen.sh &&\
./configure --prefix=/usr/local &&\
make -j && \
make install
FROM alpine:$ALPINE_VERSION
LABEL maintainer="Sergio Talens-Oliag <sto@mixinet.net>"
WORKDIR /webhook
RUN apk update &&\
apk add --no-cache ca-certificates mailcap fuse libxml2 libcurl libgcc\
libstdc++ rsync util-linux-misc &&\
rm -rf /var/cache/apk/*
COPY --from=builder /usr/local/bin/webhook /usr/local/bin/webhook
COPY --from=builder /usr/local/bin/s3fs /usr/local/bin/s3fs
COPY entrypoint.sh /
COPY hooks/* ./hooks/
EXPOSE 9000
ENTRYPOINT ["/entrypoint.sh"]
CMD ["server"]
PATCH
included on this
pull request against a released
version of the source instead of creating a fork.
The entrypoint.sh
script is used to generate the webhook
configuration file
for the existing hooks
using environment variables (basically the
WEBHOOK_WORKDIR
and the *_TOKEN
variables) and launch the webhook
service:
The entrypoint.sh
script generates the configuration file for the webhook
server calling functions that print a yaml
section for each hook
and
optionally adds rules to validate access to them comparing the value of a
X-Webhook-Token
header against predefined values.
The expected token values are taken from environment variables, we can define
a token variable for each hook
(DU_TOKEN
, HARDLINK_TOKEN
or S3_TOKEN
)
and a fallback value (COMMON_TOKEN
); if no token variable is defined for a
hook
no check is done and everybody can call it.
The Hook
Definition documentation explains the options you can use for each hook
, the
ones we have right now do the following:
du
: runs on the $WORKDIR
directory, passes as first argument to the
script the value of the path
query parameter and sets the variable
OUTPUT_FORMAT
to the fixed value json
(we use that to print the output of
the script in JSON format instead of text).hardlink
: runs on the $WORKDIR
directory and takes no parameters.s3sync
: runs on the $WORKDIR
directory and sets a lot of environment
variables from values read from the JSON encoded payload sent by the caller
(all the values must be sent by the caller even if they are assigned an empty
value, if they are missing the hook
fails without calling the script); we
also set the stream-command-output
value to true
to make the script show
its output as it is working (we patched the webhook
source to be able to
use this option).du
hook scriptThe du
hook script code checks if the argument passed is a directory,
computes its size using the du
command and prints the results in text format
or as a JSON dictionary:
hardlink
hook scriptThe hardlink
hook script is really simple, it just runs the
util-linux version of the
hardlink
command on its working directory:
We use that to reduce the size of the stored content; to manage versions of
files and folders we keep each version on a separate directory and when one or
more files are not changed this script makes them hardlinks to the same file on
disc, reducing the space used on disk.s3sync
hook scriptThe s3sync
hook script uses the s3fs
tool to mount a bucket and synchronise data between a folder inside the bucket
and a directory on the filesystem using rsync
; all values needed to execute
the task are taken from environment variables:
StatefulSet
with one replica.
Our production deployment is done on AWS and to be
able to scale we use EFS for our
PersistenVolume
; the idea is that the volume has no size limit, its
AccessMode
can be set to ReadWriteMany
and we can mount it from multiple
instances of the Pod without issues, even if they are in different availability
zones.
For development we use k3d and we are also able to scale the
StatefulSet
for testing because we use a ReadWriteOnce
PVC, but it points
to a hostPath
that is backed up by a folder that is mounted on all the
compute nodes, so in reality Pods in different k3d
nodes use the same folder
on the host.
mysecureshell
container that
can be generated using kubernetes pods as follows (we are only creating the
scs
user):
$ kubectl run "mysecureshell" --restart='Never' --quiet --rm --stdin \
--image "stodh/mysecureshell:latest" -- gen-host-keys >"./host_keys.txt"
$ kubectl run "mysecureshell" --restart='Never' --quiet --rm --stdin \
--image "stodh/mysecureshell:latest" -- gen-users-tar scs >"./users.tar"
secrets.yaml
file as follows:
$ tar xf ./users.tar user_keys.txt user_pass.txt
$ kubectl --dry-run=client -o yaml create secret generic "scs-secret" \
--from-file="host_keys.txt=host_keys.txt" \
--from-file="user_keys.txt=user_keys.txt" \
--from-file="user_pass.txt=user_pass.txt" > ./secrets.yaml
secrets.yaml
will look like the following file (the base64
would match the content of the files, of course):
statefulSet
) can be as simple as this:
On this definition we don t set the storageClassName
to use the default one.
PersistentVolume
as
required by the
Local
Persistence Volume Static Provisioner (note that the /volumes/scs-pv
has to
be created by hand, in our k3d
system we mount the same host directory on the
/volumes
path of all the nodes and create the scs-pv
directory by hand
before deploying the persistent volume):
And to make sure that everything works as expected we update the PVC definition
to add the right storageClassName
:
PersistentVolume
(we are
using the
aws-efs-csi-driver which
supports Dynamic Provisioning) but we add the storageClassName
(we set it
to the one mapped to the EFS
driver, i.e. efs-sc
) and set ReadWriteMany
as the accessMode
:
statefulSet
is as follows:
Notes about the containers:
nginx
: As this is an example the web server is not using an
AUTH_REQUEST_URI
and uses the /sftp/data
directory as the root of the web
(to get to the files uploaded for the scs
user we will need to use /scs/
as a prefix on the URLs).mysecureshell
: We are adding the IPC_OWNER
capability to the container to
be able to use some of the sftp-*
commands inside it, but they are
not really needed, so adding the capability is optional.webhook
: We are launching this container in privileged mode to be able to
use the s3fs-fuse
, as it will not work otherwise for now (see this
kubernetes issue); if
the functionality is not needed the container can be executed with regular
privileges; besides, as we are not enabling public access to this service we
don t define *_TOKEN
variables (if required the values should be read from a
Secret
object).devfuse
volume is only needed if we plan to use the s3fs
command on
the webhook
container, if not we can remove the volume definition and its
mounts.Service
object:
scs
files from the outside we can add an ingress object like
the following (the definition is for testing using the localhost
name):
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: scs-ingress
labels:
app.kubernetes.io/name: scs
spec:
ingressClassName: nginx
rules:
- host: 'localhost'
http:
paths:
- path: /scs
pathType: Prefix
backend:
service:
name: scs-svc
port:
number: 80
statefulSet
we create a namespace and apply the object
definitions shown before:
$ kubectl create namespace scs-demo
namespace/scs-demo created
$ kubectl -n scs-demo apply -f secrets.yaml
secret/scs-secrets created
$ kubectl -n scs-demo apply -f pvc.yaml
persistentvolumeclaim/scs-pvc created
$ kubectl -n scs-demo apply -f statefulset.yaml
statefulset.apps/scs created
$ kubectl -n scs-demo apply -f service.yaml
service/scs-svc created
$ kubectl -n scs-demo apply -f ingress.yaml
ingress.networking.k8s.io/scs-ingress created
kubectl
:
$ kubectl -n scs-demo get all,secrets,ingress
NAME READY STATUS RESTARTS AGE
pod/scs-0 3/3 Running 0 24s
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/scs-svc ClusterIP 10.43.0.47 <none> 22/TCP,80/TCP,9000/TCP 21s
NAME READY AGE
statefulset.apps/scs 1/1 24s
NAME TYPE DATA AGE
secret/default-token-mwcd7 kubernetes.io/service-account-token 3 53s
secret/scs-secrets Opaque 3 39s
NAME CLASS HOSTS ADDRESS PORTS AGE
ingress.networking.k8s.io/scs-ingress nginx localhost 172.21.0.5 80 17s
sftp
server from
other Pods, but to test the system we are going to do a kubectl port-forward
and connect to the server using our host client and the password we have
generated (it is on the user_pass.txt
file, inside the users.tar
archive):
$ kubectl -n scs-demo port-forward service/scs-svc 2020:22 &
Forwarding from 127.0.0.1:2020 -> 22
Forwarding from [::1]:2020 -> 22
$ PF_PID=$!
$ sftp -P 2020 scs@127.0.0.1 1
Handling connection for 2020
The authenticity of host '[127.0.0.1]:2020 ([127.0.0.1]:2020)' can't be \
established.
ED25519 key fingerprint is SHA256:eHNwCnyLcSSuVXXiLKeGraw0FT/4Bb/yjfqTstt+088.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[127.0.0.1]:2020' (ED25519) to the list of known \
hosts.
scs@127.0.0.1's password: **********
Connected to 127.0.0.1.
sftp> ls -la
drwxr-xr-x 2 sftp sftp 4096 Sep 25 14:47 .
dr-xr-xr-x 3 sftp sftp 4096 Sep 25 14:36 ..
sftp> !date -R > /tmp/date.txt 2
sftp> put /tmp/date.txt .
Uploading /tmp/date.txt to /date.txt
date.txt 100% 32 27.8KB/s 00:00
sftp> ls -l
-rw-r--r-- 1 sftp sftp 32 Sep 25 15:21 date.txt
sftp> ln date.txt date.txt.1 3
sftp> ls -l
-rw-r--r-- 2 sftp sftp 32 Sep 25 15:21 date.txt
-rw-r--r-- 2 sftp sftp 32 Sep 25 15:21 date.txt.1
sftp> put /tmp/date.txt date.txt.2 4
Uploading /tmp/date.txt to /date.txt.2
date.txt 100% 32 27.8KB/s 00:00
sftp> ls -l 5
-rw-r--r-- 2 sftp sftp 32 Sep 25 15:21 date.txt
-rw-r--r-- 2 sftp sftp 32 Sep 25 15:21 date.txt.1
-rw-r--r-- 1 sftp sftp 32 Sep 25 15:21 date.txt.2
sftp> exit
$ kill "$PF_PID"
[1] + terminated kubectl -n scs-demo port-forward service/scs-svc 2020:22
sftp
service on the forwarded port with the scs
user.date.txt
file from the
URL http://localhost/scs/date.txt:
$ curl -s http://localhost/scs/date.txt
Sun, 25 Sep 2022 17:21:51 +0200
hooks
directly,
from a CronJob
and from a Job
.
du
)In our deployment the direct calls are done from other Pods, to simulate it we
are going to do a port-forward
and call the script with an existing PATH (the
root directory) and a bad one:
$ kubectl -n scs-demo port-forward service/scs-svc 9000:9000 >/dev/null &
$ PF_PID=$!
$ JSON="$(curl -s "http://localhost:9000/hooks/du?path=.")"
$ echo $JSON
"path":"","bytes":"4160"
$ JSON="$(curl -s "http://localhost:9000/hooks/du?path=foo")"
$ echo $JSON
"error":"The provided PATH ('foo') is not a directory"
$ kill $PF_PID
.
PATH and the output is in json
format because we export OUTPUT_FORMAT
with
the value json
on the webhook
configuration.hardlink
)As explained before, the webhook
container can be used to run cronjobs
; the
following one uses an alpine
container to call the hardlink
script each
minute (that setup is for testing, obviously):
The following console session shows how we create the object, allow a couple of
executions and remove it (in production we keep it running but once a day, not
each minute):
$ kubectl -n scs-demo apply -f webhook-cronjob.yaml 1
cronjob.batch/hardlink created
$ kubectl -n scs-demo get pods -l "cronjob=hardlink" -w 2
NAME READY STATUS RESTARTS AGE
hardlink-27735351-zvpnb 0/1 Pending 0 0s
hardlink-27735351-zvpnb 0/1 ContainerCreating 0 0s
hardlink-27735351-zvpnb 0/1 Completed 0 2s
^C
$ kubectl -n scs-demo logs pod/hardlink-27735351-zvpnb 3
Mode: real
Method: sha256
Files: 3
Linked: 1 files
Compared: 0 xattrs
Compared: 1 files
Saved: 32 B
Duration: 0.000220 seconds
$ sleep 60
$ kubectl -n scs-demo get pods -l "cronjob=hardlink" 4
NAME READY STATUS RESTARTS AGE
hardlink-27735351-zvpnb 0/1 Completed 0 83s
hardlink-27735352-br5rn 0/1 Completed 0 23s
$ kubectl -n scs-demo logs pod/hardlink-27735352-br5rn 5
Mode: real
Method: sha256
Files: 3
Linked: 0 files
Compared: 0 xattrs
Compared: 0 files
Saved: 0 B
Duration: 0.000070 seconds
$ kubectl -n scs-demo delete -f webhook-cronjob.yaml 6
cronjob.batch "hardlink" deleted
cronjob
label, we interrupt it once we see
that the first run has been completed.date.txt.2
has been replaced by a hardlink (the
summary does not name the file, but it is the only option knowing the
contents from the original upload).s3sync
)The following job can be used to synchronise the contents of a directory in a
S3 bucket with the SCS Filesystem:
The file with parameters for the script must be something like this:
Once we have both files we can run the Job as follows:
$ kubectl -n scs-demo create secret generic webhook-job-secrets \ 1
--from-file="s3sync.json=s3sync.json"
secret/webhook-job-secrets created
$ kubectl -n scs-demo apply -f webhook-job.yaml 2
job.batch/s3sync created
$ kubectl -n scs-demo get pods -l "cronjob=s3sync" 3
NAME READY STATUS RESTARTS AGE
s3sync-zx2cj 0/1 Completed 0 12s
$ kubectl -n scs-demo logs s3sync-zx2cj 4
Mounted bucket 's3fs-test' on '/root/tmp.jiOjaF/s3data'
sending incremental file list
created directory ./test
./
kyso.png
Number of files: 2 (reg: 1, dir: 1)
Number of created files: 2 (reg: 1, dir: 1)
Number of deleted files: 0
Number of regular files transferred: 1
Total file size: 15,075 bytes
Total transferred file size: 15,075 bytes
Literal data: 15,075 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.147 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 15,183
Total bytes received: 74
sent 15,183 bytes received 74 bytes 30,514.00 bytes/sec
total size is 15,075 speedup is 0.99
Called umount for '/root/tmp.jiOjaF/s3data'
Script exit code: 0
$ kubectl -n scs-demo delete -f webhook-job.yaml 5
job.batch "s3sync" deleted
$ kubectl -n scs-demo delete secrets webhook-job-secrets 6
secret "webhook-job-secrets" deleted
webhook-job-secrets
secret that contains the
s3sync.json
file.cronjob=s3sync
we get the Pods executed by the job.Resolved disagreements and personality clashes result in greater intimacy, and a spirit of co-operation emerges.Teams need to understand these stages because a team can regress to earlier stages when its composition or goals change. A new member, the departure of an existing member, changes in supervisor or leadership style can all lead a team to regress to the storming stage and fail to perform for a time. When you see a team member say this, as I observed in an IRC channel recently, you know the team is performing:
nice teamwork these busy days Seen on IRC in the channel of a performing teamTuckman s model describes a team s performance overall, but how can team members establish what they can contribute and how can they go doing so confidently and effectively? Belbin s Team Roles
The types of behaviour in which people engage are infinite. But the range of useful behaviours, which make an effective contribution to team performance, is finite. These behaviours are grouped into a set number of related clusters, to which the term Team Role is applied. Belbin, R M. Team Roles at Work. Oxford: Butterworth-Heinemann, 2010Dr Meredith Belbin s thesis, based on nearly ten years research during the 1970s and 1980s, is that each team has a number of roles which need to be filled at various times, but they re not innate characteristics of the people filling them. People may have attributes which make them more or less suited to each role, and they can consciously take up a role if they recognise its need in the team at a particular time. Belbin s nine team roles are:
npm
packages on our gitlab
instance npm registry
and even the publication into the npmjs registry.
To publish the packages we have added rules to the gitlab-ci
configuration of
the relevant repositories and we publish them when a tag
is created.
As the we are lazy by definition, I configured the system to use the tag
as
the package version; I tested if the contents of the package.json
where in
sync with the expected version and if it was not I updated it and did a force
push of the tag
with the updated file using the following code on the script
that publishes the package:
# Update package version & add it to the .build-args
INITIAL_PACKAGE_VERSION="$(npm pkg get version tr -d '"')"
npm version --allow-same --no-commit-hooks --no-git-tag-version \
"$CI_COMMIT_TAG"
UPDATED_PACKAGE_VERSION="$(npm pkg get version tr -d '"')"
echo "UPDATED_PACKAGE_VERSION=$UPDATED_PACKAGE_VERSION" >> .build-args
# Update tag if the version was updated or abort
if [ "$INITIAL_PACKAGE_VERSION" != "$UPDATED_PACKAGE_VERSION" ]; then
if [ -n "$CI_GIT_USER" ] && [ -n "$CI_GIT_TOKEN" ]; then
git commit -m "Updated version from tag $CI_COMMIT_TAG" package.json
git tag -f "$CI_COMMIT_TAG" -m "Updated version from tag"
git push -f -o ci.skip origin "$CI_COMMIT_TAG"
else
echo "!!! ERROR !!!"
echo "The updated tag could not be uploaded."
echo "Set CI_GIT_USER and CI_GIT_TOKEN or fix the 'package.json' file"
echo "!!! ERROR !!!"
exit 1
fi
fi
branch
using the
tag
and update it, but I drop the idea pretty soon as there were multiple
issues to consider (i.e. we can have tags pointing to commits present in
multiple branches and even if it only points to one the tag
does not have to
be the HEAD
of the branch making the inclusion difficult).
In any case this system was working, so we left it until we started to publish
to the NPM Registry; as we are using a token to push the packages that we
don t want all developers to have access to (right now it would not matter, but
when the team grows it will) I started to use gitlab
protected
branches on the projects that need it and adjusting the .npmrc
file using
protected
variables.
The problem then was that we can no longer do a standard force push for a
branch (that is the main point of the protected branches feature) unless we
use the gitlab api, so the tags with the wrong version started to fail.
As the way things were being done seemed dirty anyway I thought that the best
way of fixing things was to forbid users to push a tag
that includes a
version that does not match the package.json
version.
After thinking about it we decided to use
githooks on the
gitlab server for
the repositories that need it, as we are only interested in tags
we are going
to use the update hook; it is
executed once for each ref to be updated, and takes three parameters:
gitaly
relative path of each repo and
located it on the server filesystem (as I said we are using docker
and the
gitlab s data
directory is on /srv/gitlab/data
, so the path to the repo has
the form /srv/gitlab/data/git-data/repositories/@hashed/xx/yy/hash.git
).
Once we have the directory we need to:
custom_hooks
sub directory inside it,update
script (as we only need one script we used that instead of
creating an update.d
directory, the good thing is that this will also work
with a standard git server renaming the base directory to hooks
instead of
custom_hooks
),$ cd /srv/gitlab/data/git-data/repositories/@hashed/xx/yy/hash.git
$ mkdir custom_hooks
$ edit_or_copy custom_hooks/update
$ chmod 0755 custom_hooks/update
$ chown --reference=. -R custom_hooks
update
script we are using is as follows:
#!/bin/sh
set -e
# kyso update hook
#
# Right now it checks version.txt or package.json versions against the tag name
# (it supports a 'v' prefix on the tag)
# Arguments
ref_name="$1"
old_rev="$2"
new_rev="$3"
# Initial test
if [ -z "$ref_name" ] [ -z "$old_rev" ] [ -z "$new_rev" ]; then
echo "usage: $0 <ref> <oldrev> <newrev>" >&2
exit 1
fi
# Get the tag short name
tag_name="$ ref_name##refs/tags/ "
# Exit if the update is not for a tag
if [ "$tag_name" = "$ref_name" ]; then
exit 0
fi
# Get the null rev value (string of zeros)
zero=$(git hash-object --stdin </dev/null tr '0-9a-f' '0')
# Get if the tag is new or not
if [ "$old_rev" = "$zero" ]; then
new_tag="true"
else
new_tag="false"
fi
# Get the type of revision:
# - delete: if the new_rev is zero
# - commit: annotated tag
# - tag: un-annotated tag
if [ "$new_rev" = "$zero" ]; then
new_rev_type="delete"
else
new_rev_type="$(git cat-file -t "$new_rev")"
fi
# Exit if we are deleting a tag (nothing to check here)
if [ "$new_rev_type" = "delete" ]; then
exit 0
fi
# Check the version against the tag (supports version.txt & package.json)
if git cat-file -e "$new_rev:version.txt" >/dev/null 2>&1; then
version="$(git cat-file -p "$new_rev:version.txt")"
if [ "$version" = "$tag_name" ] [ "$version" = "$ tag_name#v " ]; then
exit 0
else
EMSG="tag '$tag_name' and 'version.txt' contents '$version' don't match"
echo "GL-HOOK-ERR: $EMSG"
exit 1
fi
elif git cat-file -e "$new_rev:package.json" >/dev/null 2>&1; then
version="$(
git cat-file -p "$new_rev:package.json" jsonpath version tr -d '\[\]"'
)"
if [ "$version" = "$tag_name" ] [ "$version" = "$ tag_name#v " ]; then
exit 0
else
EMSG="tag '$tag_name' and 'package.json' version '$version' don't match"
echo "GL-HOOK-ERR: $EMSG"
exit 1
fi
else
# No version.txt or package.json file found
exit 0
fi
tags
, if the ref_name
does not have the prefix
refs/tags/
the script does an exit 0
,tag
is new or not we are not using the
value (in gitlab
that is handled by the protected tag feature),tag
the script does an exit 0
, we don t need to
check anything in that case,tag
is annotated or not (we set the new_rev_type
to
tag
or commit
, but we don t use the value),version.txt
file and if it does not exist we check the
package.json
file, if it does not exist either we do an exit 0
, as there
is no version to check against and we allow that on a tag,GL-HOOK-ERR:
prefix to the messages to show them on the gitlab
web interface (can be tested creating a tag from it),version
on the package.json
file we use the jsonpath
binary
(it is installed by the jsonpath ruby
gem) because it is available on the gitlab
container (initially I used
sed
to get the value, but a real JSON parser is always a better option).tag
to a repository
that has a version.txt
file or package.json
file and the tag does not match
the version (if version.txt
is present it takes precedence) the push
fails.
If the tag
matches or the files are not present the tag
is added if the
user has permission to add it in gitlab
(our hook is only executed if the
user is allowed to create or update the tag
).
Series: | Fractured Fables #2 |
Publisher: | Tordotcom |
Copyright: | 2022 |
ISBN: | 1-250-76665-6 |
Format: | Kindle |
Pages: | 129 |
In truth, the histories of Arpanet and BBS networks were interwoven socially and materially as ideas, technologies, and people flowed between them. The history of the internet could be a thrilling tale inclusive of many thousands of networks, big and small, urban and rural, commercial and voluntary. Instead, it is repeatedly reduced to the story of the singular Arpanet.Kevin Driscoll goes on to highlight the social aspects of the modem world , how BBSs and online services like AOL and CompuServe were ways for people to connect. And yet, AOL members couldn t easily converse with CompuServe members, and vice-versa. Sound familiar?
Today s social media ecosystem functions more like the modem world of the late 1980s and early 1990s than like the open social web of the early 21st century. It is an archipelago of proprietary platforms, imperfectly connected at their borders. Any gateways that do exist are subject to change at a moment s notice. Worse, users have little recourse, the platforms shirk accountability, and states are hesitant to intervene.Yes, it does. As he adds, People aren t the problem. The problem is the platforms. A thought-provoking article, and I think I ll need to buy the book it s excerpted from!
gpg -q -d zstdcat -T0 zfs receive -u -o readonly=on "$STORE/$DEST"This processes tens of thousands of zfs sends per week. Recently, having written Filespooler, I switched to sending the backups using Filespooler over NNCP. Now fspl (the Filespooler executable) opens the file for each stream and then connects it to what amounts to this pipeline:
bash -c 'gpg -q -d 2>/dev/null zstdcat -T0' zfs receive -u -o readonly=on "$STORE/$DEST"Actually, to be more precise, it spins up the bash part of it, reads a few bytes from it, and then connects it to the zfs receive. And this works well almost always. In something like 1/1000 of the cases, it deadlocks, and I still don t know why. But I can talk about the journey of trying to figure it out (and maybe some of you will have some ideas). Filespooler is written in Rust, and uses Rust s Command system. Effectively what happens is this:
int err = zfs_file_read(fp, (char *)buf + done, len - done, &resid); if (resid == len - done) /* * Note: ECKSUM or ZFS_ERR_STREAM_TRUNCATED indicates * that the receive was interrupted and can * potentially be resumed. */ err = SET_ERROR(ZFS_ERR_STREAM_TRUNCATED);resid is an output parameter with the number of bytes remaining from a short read, so in this case, if the read produced zero bytes, then it sets that error. What s zfs_file_read then? It boils down to a thin wrapper around kernel_read(). This winds up calling __kernel_read(), which calls read_iter on the pipe, which is pipe_read(). That s where I don t have the knowledge to get into the weeds right now. So it seems likely to me that the problem has something to do with zfs receive. But, what, and why does it only not work in this one very specific situation, and only so rarely? And why does attaching strace to zstdcat make it all work again? I m indeed puzzled! Update 2022-06-20: See the followup post which identifies this as likely a kernel bug and explains why this particular use of Filespooler made it easier to trigger.
hugo
, asciidoctor
and the papermod
theme, how I publish it using
nginx
, how I ve integrated the remark42
comment system and how I ve
automated its publication using gitea
and json2file-go
.
It is a long post, but I hope that at least parts of it can be interesting for
some, feel free to ignore it if that is not your case
config.yml
file is the one shown below (probably some of the
settings are not required nor being used right now, but I m including the
current file, so this post will have always the latest version of it):
Some notes about the settings:
disableHLJS
and assets.disableHLJS
are set to true
; we plan to use
rouge
on adoc
and the inclusion of the hljs
assets adds styles that
collide with the ones used by rouge
.ShowToc
is set to true
and the TocOpen
setting is set to false
to
make the ToC appear collapsed initially. My plan was to use the asciidoctor
ToC, but after trying I believe that the theme one looks nice and I don t
need to adjust styles, although it has some issues with the html5s
processor (the admonition titles use <h6>
and they are shown on the ToC,
which is weird), to fix it I ve copied the layouts/partial/toc.html
to my
site repository and replaced the range of headings to end at 5
instead of
6
(in fact 5
still seems a lot, but as I don t think I ll use that heading
level on the posts it doesn t really matter).params.profileMode
values are adjusted, but for now I ve left it disabled
setting params.profileMode.enabled
to false
and I ve set the
homeInfoParams
to show more or less the same content with the latest posts
under it (I ve added some styles to my custom.css
style sheet to center the
text and image of the first post to match the look and feel of the profile).asciidocExt
section I ve adjusted the backend
to use html5s
,
I ve added the asciidoctor-html5s
and asciidoctor-diagram
extensions to
asciidoctor
and adjusted the workingFolderCurrent
to true
to make
asciidoctor-diagram
work right (haven t tested it yet).asciidoctor
using the html5s
processor I ve added some files to
the assets/css/extended
directory:
assets/css/extended/custom.css
to
make the homeInfoParams
look like the profile page and I ve also changed a
little bit some theme styles to make things look better with the html5s
output:assets/css/extended/adoc.css
with some styles
taken from the asciidoctor-default.css
, see this
blog
post about the original file; mine is the same after formatting it with
css-beautify and editing it to use variables for
the colors to support light and dark themes:theme-vars.css
file that changes the highlighted code background color and adds the color
definitions used by the admonitions:font-awesome
, so I ve downloaded its resources for
version 4.7.0
(the one used by asciidoctor
) storing the
font-awesome.css
into on the assets/css/extended
dir (that way it is
merged with the rest of .css
files) and copying the fonts to the
static/assets/fonts/
dir (will be served directly):FA_BASE_URL="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0"
curl "$FA_BASE_URL/css/font-awesome.css" \
> assets/css/extended/font-awesome.css
for f in FontAwesome.otf fontawesome-webfont.eot \
fontawesome-webfont.svg fontawesome-webfont.ttf \
fontawesome-webfont.woff fontawesome-webfont.woff2; do
curl "$FA_BASE_URL/fonts/$f" > "static/assets/fonts/$f"
done
css
compatible with rouge
) so we need a css
to do the highlight styling; as
rouge
provides a way to export them, I ve created the
assets/css/extended/rouge.css
file with the thankful_eyes
theme:rougify style thankful_eyes > assets/css/extended/rouge.css
html5s
backend with admonitions I ve added a
variation of the example found on this
blog
post to assets/js/adoc-admonitions.js
:
and enabled its minified use on the layouts/partials/extend_footer.html
file
adding the following lines to it:
- $admonitions := slice (resources.Get "js/adoc-admonitions.js")
resources.Concat "assets/js/adoc-admonitions.js" minify fingerprint
<script defer crossorigin="anonymous" src=" $admonitions.RelPermalink "
integrity=" $admonitions.Data.Integrity "></script>
layouts/partials/comments.html
with the following content
based on the remark42
documentation, including extra code to sync the dark/light setting with the
one set on the site:
In development I use it with anonymous comments enabled, but to avoid SPAM
the production site uses social logins (for now I ve only enabled Github
& Google, if someone requests additional services I ll check them, but those
were the easy ones for me initially).
To support theme switching with remark42
I ve also added the following inside
the layouts/partials/extend_footer.html
file:
- if (not site.Params.disableThemeToggle)
<script>
/* Function to change theme when the toggle button is pressed */
document.getElementById("theme-toggle").addEventListener("click", () =>
if (typeof window.REMARK42 != "undefined")
if (document.body.className.includes('dark'))
window.REMARK42.changeTheme('light');
else
window.REMARK42.changeTheme('dark');
);
</script>
- end
theme-toggle
button is pressed we change the remark42
theme before the PaperMod
one (that s needed here only, on page loads the
remark42
theme is synced with the main one using the code from the
layouts/partials/comments.html
shown earlier).docker-compose
with the following
configuration:
To run it properly we have to create the .env
file with the current user ID
and GID on the variables APP_UID
and APP_GID
(if we don t do it the files
can end up being owned by a user that is not the same as the one running the
services):
$ echo "APP_UID=$(id -u)\nAPP_GID=$(id -g)" > .env
Dockerfile
used to generate the sto/hugo-adoc
is:
If you review it you will see that I m using the
docker-asciidoctor image as
the base; the idea is that this image has all I need to work with asciidoctor
and to use hugo
I only need to download the binary from their latest release
at github (as we are using an
image based on alpine we also need to install the
libc6-compat
package, but once that is done things are working fine for me so
far).
The image does not launch the server by default because I don t want it to; in
fact I use the same docker-compose.yml
file to publish the site in production
simply calling the container without the arguments passed on the
docker-compose.yml
file (see later).
When running the containers with docker-compose up
(or docker compose up
if
you have the docker-compose-plugin
package installed) we also launch a nginx
container and the remark42
service so we can test everything together.
The Dockerfile
for the remark42
image is the original one with an updated
version of the init.sh
script:
The updated init.sh
is similar to the original, but allows us to use an
APP_GID
variable and updates the /etc/group
file of the container so the
files get the right user and group (with the original script the group is
always 1001
):
The environment file used with remark42
for development is quite minimal:
And the nginx/default.conf
file used to publish the service locally is simple
too:
main
Debian repository:
git
to clone & pull the repository,jq
to parse json
files from shell scripts,json2file-go
to save the webhook messages to files,inotify-tools
to detect when new files are stored by json2file-go
and
launch scripts to process them,nginx
to publish the site using HTTPS and work as proxy for
json2file-go
and remark42
(I run it using a container),task-spool
to queue the scripts that update the deployment.docker
and docker compose
from the debian packages on the
docker
repository:
docker-ce
to run the containers,docker-compose-plugin
to run docker compose
(it is a plugin, so no -
in
the name).git
repository I ve created a deploy key, added it to gitea
and cloned the project on the /srv/blogops
PATH (that route is owned by a
regular user that has permissions to run docker
, as I said before).hugo
To compile the site we are using the docker-compose.yml
file seen before, to
be able to run it first we build the container images and once we have them we
launch hugo
using docker compose run
:
$ cd /srv/blogops
$ git pull
$ docker compose build
$ if [ -d "./public" ]; then rm -rf ./public; fi
$ docker compose run hugo --
/srv/blogops/public
(we remove the
directory first because hugo
does not clean the destination folder as
jekyll
does).
The deploy script re-generates the site as described and moves the public
directory to its final place for publishing.remark42
with dockerOn the /srv/blogops/remark42
folder I have the following docker-compose.yml
:
The ../.env
file is loaded to get the APP_UID
and APP_GID
variables that
are used by my version of the init.sh
script to adjust file permissions and
the env.prod
file contains the rest of the settings for remark42
, including
the social network tokens (see the
remark42 documentation for
the available parameters, I don t include my configuration here because some of
them are secrets).nginx
configuration for the blogops.mixinet.net
site is as simple as:
server
listen 443 ssl http2;
server_name blogops.mixinet.net;
ssl_certificate /etc/letsencrypt/live/blogops.mixinet.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/blogops.mixinet.net/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
access_log /var/log/nginx/blogops.mixinet.net-443.access.log;
error_log /var/log/nginx/blogops.mixinet.net-443.error.log;
root /srv/blogops/nginx/public_html;
location /
try_files $uri $uri/ =404;
include /srv/blogops/nginx/remark42.conf;
server
listen 80 ;
listen [::]:80 ;
server_name blogops.mixinet.net;
access_log /var/log/nginx/blogops.mixinet.net-80.access.log;
error_log /var/log/nginx/blogops.mixinet.net-80.error.log;
if ($host = blogops.mixinet.net)
return 301 https://$host$request_uri;
return 404;
/srv/blogops/nginx/public_html
and not on /srv/blogops/public
; the reason
for that is that I want to be able to compile without affecting the running
site, the deployment script generates the site on /srv/blogops/public
and if
all works well we rename folders to do the switch, making the change feel almost
atomic.gitea
at my home and the VM where the blog is served, I m
going to configure the json2file-go
to listen for connections on a high port
using a self signed certificate and listening on IP addresses only reachable
through the VPN.
To do it we create a systemd socket
to run json2file-go
and adjust its
configuration to listen on a private IP (we use the FreeBind
option on its
definition to be able to launch the service even when the IP is not available,
that is, when the VPN is down).
The following script can be used to set up the json2file-go
configuration:
mkcert
to create the temporary certificates, to install the
package on bullseye
the backports
repository must be available.json2file-go
server we go to the project and enter into
the hooks/gitea/new
page, once there we create a new webhook of type gitea
and set the target URL to https://172.31.31.1:4443/blogops
and on the secret
field we put the token generated with uuid
by the setup script:
sed -n -e 's/blogops://p' /etc/json2file-go/dirlist
*
webhook
section of the app.ini
of our gitea
server allows us to call the IP and skips the TLS verification (you can see the
available options on the
gitea
documentation).
The [webhook]
section of my server looks like this:
[webhook]
ALLOWED_HOST_LIST=private
SKIP_TLS_VERIFY=true
webhook
configured we can try it and if it works our
json2file
server will store the file on the
/srv/blogops/webhook/json2file/blogops/
folder.gitea
and store the messages on files, but we have to do something to
process those files once they are saved in our machine.
An option could be to use a cronjob
to look for new files, but we can do
better on Linux using inotify
we will use the inotifywait
command from
inotify-tools
to watch the json2file
output directory and execute a script
each time a new file is moved inside it or closed after writing
(IN_CLOSE_WRITE
and IN_MOVED_TO
events).
To avoid concurrency problems we are going to use task-spooler
to launch the
scripts that process the webhooks using a queue of length 1, so they are
executed one by one in a FIFO queue.
The spooler script is this:
To run it as a daemon we install it as a systemd service
using the following
script:
hugo
with docker
compose
,Publisher: | Subterranean Press |
Copyright: | 2020 |
ISBN: | 1-59606-992-9 |
Format: | Kindle |
Pages: | 111 |
"You are displaying a very small-minded attitude," said the fairy, who seemed genuinely grieved by this. "Consider the orange-peel, which by itself has many very nice properties. Now, if you had a more educated brain (I cannot consider myself educated; I have only attempted to better my situation) you would have immediately said, 'Why, if I had some liquor, or even very hot water, I could extract some oil from this orange-peel, which as everyone knows is antibacterial; that may well do my hands some good,' and you wouldn't be in such a stupid predicament."On balance, I think this style worked. It occasionally annoyed me, but it has some charm. About halfway through, I was finding the story lightly entertaining, although I would have preferred a bit less grime, illness, and physical injury. Unfortunately, the rest of the story didn't work for me. The dynamic between Floralinda and Cobweb turns into a sort of D&D progression through monster fights, and while there are some creative twists to those fights, they become all of a sameness. And while I won't spoil the ending, it didn't work for me. I think I see what Muir was trying to do, and I have some intellectual appreciation for the idea, but it wasn't emotionally satisfying. I think my root problem with this story is that Muir sets up a rather interesting world, one in which witches artistically imprison princesses, and particularly bright princesses (with the help of amateur chemist fairies) can use the trappings of a magical tower in ways the witch never intended. I liked that; it has a lot of potential. But I didn't feel like that potential went anywhere satisfying. There is some relationship and characterization work, and it reached some resolution, but it didn't go as far as I wanted. And, most significantly, I found the end point the characters reached in relation to the world to be deeply unsatisfying and vaguely irritating. I wanted to like this more than I did. I think there's a story idea in here that I would have enjoyed more. Unfortunately, it's not the one that Muir wrote, and since so much of my problem is with the ending, I can't provide much guidance on whether someone else would like this story better (and why). But if the idea of taking apart a fairy-tale tower and repurposing the pieces sounds appealing, and if you get along better with Muir's illness motif than I do, you may enjoy this more than I did. Rating: 5 out of 10
Freexian s mission is to help Debian evolve to be the leading Linux distribution and a model to follow in the free software world.We want to achieve that by enabling passionate contributors to spend most of their time working on Debian and free software, combining lucrative work in support of enterprise customers, core contributions to Debian s processes, and personal goals.Extract from Freexian s mission statementIf Debian is an important part of your life, if improving Debian is something that you enjoy doing, if you have enough skills and Debian expertise to match some of the roles that we have described on our Join us page, then we would love to hear from you at gerants@freexian.com, so we can figure out a way to work together.
Dogtooth (2009) A father, a mother, a brother and two sisters live in a large and affluent house behind a very high wall and an always-locked gate. Only the father ever leaves the property, driving to the factory that he happens to own. Dogtooth goes far beyond any allusion to Josef Fritzl's cellar, though, as the children's education is a grotesque parody of home-schooling. Here, the parents deliberately teach their children the wrong meaning of words (e.g. a yellow flower is called a 'zombie'), all of which renders the outside world utterly meaningless and unreadable, and completely mystifying its very existence. It is this creepy strangeness within a 'regular' family unit in Dogtooth that is both socially and epistemically horrific, and I'll say nothing here of its sexual elements as well. Despite its cold, inscrutable and deadpan surreality, Dogtooth invites all manner of potential interpretations. Is this film about the artificiality of the nuclear family that the West insists is the benchmark of normality? Or is it, as I prefer to believe, something more visceral altogether: an allegory for the various forms of ontological violence wrought by fascism, as well a sobering nod towards some of fascism's inherent appeals? (Perhaps it is both. In 1972, French poststructuralists Gilles and F lix Guattari wrote Anti-Oedipus, which plays with the idea of the family unit as a metaphor for the authoritarian state.) The Greek-language Dogtooth, elegantly shot, thankfully provides no easy answers.
Holy Motors (2012) There is an infamous scene in Un Chien Andalou, the 1929 film collaboration between Luis Bu uel and famed artist Salvador Dal . A young woman is cornered in her own apartment by a threatening man, and she reaches for a tennis racquet in self-defence. But the man suddenly picks up two nearby ropes and drags into the frame two large grand pianos... each leaden with a dead donkey, a stone tablet, a pumpkin and a bewildered priest. This bizarre sketch serves as a better introduction to Leos Carax's Holy Motors than any elementary outline of its plot, which ostensibly follows 24 hours in the life of a man who must play a number of extremely diverse roles around Paris... all for no apparent reason. (And is he even a man?) Surrealism as an art movement gets a pretty bad wrap these days, and perhaps justifiably so. But Holy Motors and Un Chien Andalou serve as a good reminder that surrealism can be, well, 'good, actually'. And if not quite high art, Holy Motors at least demonstrates that surrealism can still unnerving and hilariously funny. Indeed, recalling the whimsy of the plot to a close friend, the tears of laughter came unbidden to my eyes once again. ("And then the limousines...!") Still, it is unclear how Holy Motors truly refreshes surrealism for the twenty-first century. Surrealism was, in part, a reaction to the mechanical and unfeeling brutality of World War I and ultimately sought to release the creative potential of the unconscious mind. Holy Motors cannot be responding to another continental conflagration, and so it appears to me to be some kind of commentary on the roles we exhibit in an era of 'post-postmodernity': a sketch on our age of performative authenticity, perhaps, or an idle doodle on the function and psychosocial function of work. Or perhaps not. After all, this film was produced in a time that offers the near-universal availability of mind-altering substances, and this certainly changes the context in which this film was both created. And, how can I put it, was intended to be watched.
Manchester by the Sea (2016) An absolutely devastating portrayal of a character who is unable to forgive himself and is hesitant to engage with anyone ever again. It features a near-ideal balance between portraying unrecoverable anguish and tender warmth, and is paradoxically grandiose in its subtle intimacy. The mechanics of life led me to watch this lying on a bed in a chain hotel by Heathrow Airport, and if this colourless circumstance blunted the film's emotional impact on me, I am probably thankful for it. Indeed, I find myself reduced in this review to fatuously recalling my favourite interactions instead of providing any real commentary. You could write a whole essay about one particular incident: its surfaces, subtexts and angles... all despite nothing of any substance ever being communicated. Truly stunning.
McCabe & Mrs. Miller (1971) Roger Ebert called this movie one of the saddest films I have ever seen, filled with a yearning for love and home that will not ever come. But whilst it is difficult to disagree with his sentiment, Ebert's choice of sad is somehow not quite the right word. Indeed, I've long regretted that our dictionaries don't have more nuanced blends of tragedy and sadness; perhaps the Ancient Greeks can loan us some. Nevertheless, the plot of this film is of a gambler and a prostitute who become business partners in a new and remote mining town called Presbyterian Church. However, as their town and enterprise booms, it comes to the attention of a large mining corporation who want to bully or buy their way into the action. What makes this film stand out is not the plot itself, however, but its mood and tone the town and its inhabitants seem to be thrown together out of raw lumber, covered alternatively in mud or frozen ice, and their days (and their personalities) are both short and dark in equal measure. As a brief aside, if you haven't seen a Roger Altman film before, this has all the trappings of being a good introduction. As Ebert went on to observe: This is not the kind of movie where the characters are introduced. They are all already here. Furthermore, we can see some of Altman's trademark conversations that overlap, a superb handling of ensemble casts, and a quietly subversive view of the tyranny of 'genre'... and the latter in a time when the appetite for revisionist portrays of the West was not very strong. All of these 'Altmanian' trademarks can be ordered in much stronger measures in his later films: in particular, his comedy-drama Nashville (1975) has 24 main characters, and my jejune interpretation of Gosford Park (2001) is that it is purposefully designed to poke fun those who take a reductionist view of 'genre', or at least on the audience's expectations. (In this case, an Edwardian-era English murder mystery in the style of Agatha Christie, but where no real murder or detection really takes place.) On the other hand, McCabe & Mrs. Miller is actually a poor introduction to Altman. The story is told in a suitable deliberate and slow tempo, and the two stars of the film are shown thoroughly defrocked of any 'star status', in both the visual and moral dimensions. All of these traits are, however, this film's strength, adding up to a credible, fascinating and riveting portrayal of the old West.
Detour (1945) Detour was filmed in less than a week, and it's difficult to decide out of the actors and the screenplay which is its weakest point.... Yet it still somehow seemed to drag me in. The plot revolves around luckless Al who is hitchhiking to California. Al gets a lift from a man called Haskell who quickly falls down dead from a heart attack. Al quickly buries the body and takes Haskell's money, car and identification, believing that the police will believe Al murdered him. An unstable element is soon introduced in the guise of Vera, who, through a set of coincidences that stretches credulity, knows that this 'new' Haskell (ie. Al pretending to be him) is not who he seems. Vera then attaches herself to Al in order to blackmail him, and the world starts to spin out of his control. It must be understood that none of this is executed very well. Rather, what makes Detour so interesting to watch is that its 'errors' lend a distinctively creepy and unnatural hue to the film. Indeed, in the early twentieth century, Sigmund Freud used the word unheimlich to describe the experience of something that is not simply mysterious, but something creepy in a strangely familiar way. This is almost the perfect description of watching Detour its eerie nature means that we are not only frequently second-guessed about where the film is going, but are often uncertain whether we are watching the usual objective perspective offered by cinema. In particular, are all the ham-fisted segues, stilted dialogue and inscrutable character motivations actually a product of Al inventing a story for the viewer? Did he murder Haskell after all, despite the film 'showing' us that Haskell died of natural causes? In other words, are we watching what Al wants us to believe? Regardless of the answers to these questions, the film succeeds precisely because of its accidental or inadvertent choices, so it is an implicit reminder that seeking the director's original intention in any piece of art is a complete mirage. Detour is certainly not a good film, but it just might be a great one. (It is a short film too, and, out of copyright, it is available online for free.)
Safe (1995) Safe is a subtly disturbing film about an upper-middle-class housewife who begins to complain about vague symptoms of illness. Initially claiming that she doesn't feel right, Carol starts to have unexplained headaches, a dry cough and nosebleeds, and eventually begins to have trouble breathing. Carol's family doctor treats her concerns with little care, and suggests to her husband that she sees a psychiatrist. Yet Carol's episodes soon escalate. For example, as a 'homemaker' and with nothing else to occupy her, Carol's orders a new couch for a party. But when the store delivers the wrong one (although it is not altogether clear that they did), Carol has a near breakdown. Unsure where to turn, an 'allergist' tells Carol she has "Environmental Illness," and so Carol eventually checks herself into a new-age commune filled with alternative therapies. On the surface, Safe is thus a film about the increasing about of pesticides and chemicals in our lives, something that was clearly felt far more viscerally in the 1990s. But it is also a film about how lack of genuine healthcare for women must be seen as a critical factor in the rise of crank medicine. (Indeed, it made for something of an uncomfortable watch during the coronavirus lockdown.) More interestingly, however, Safe gently-yet-critically examines the psychosocial causes that may be aggravating Carol's illnesses, including her vacant marriage, her hollow friends and the 'empty calorie' stimulus of suburbia. None of this should be especially new to anyone: the gendered Victorian term 'hysterical' is often all but spoken throughout this film, and perhaps from the very invention of modern medicine, women's symptoms have often regularly minimised or outright dismissed. (Hilary Mantel's 2003 memoir, Giving Up the Ghost is especially harrowing on this.) As I opened this review, the film is subtle in its messaging. Just to take one example from many, the sound of the cars is always just a fraction too loud: there's a scene where a group is eating dinner with a road in the background, and the total effect can be seen as representing the toxic fumes of modernity invading our social lives and health. I won't spoiler the conclusion of this quietly devasting film, but don't expect a happy ending.
The Driver (1978) Critics grossly misunderstood The Driver when it was first released. They interpreted the cold and unemotional affect of the characters with the lack of developmental depth, instead of representing their dissociation from the society around them. This reading was encouraged by the fact that the principal actors aren't given real names and are instead known simply by their archetypes instead: 'The Driver', 'The Detective', 'The Player' and so on. This sort of quasi-Jungian erudition is common in many crime films today (Reservoir Dogs, Kill Bill, Layer Cake, Fight Club), so the critics' misconceptions were entirely reasonable in 1978. The plot of The Driver involves the eponymous Driver, a noted getaway driver for robberies in Los Angeles. His exceptional talent has far prevented him from being captured thus far, so the Detective attempts to catch the Driver by pardoning another gang if they help convict the Driver via a set-up robbery. To give himself an edge, however, The Driver seeks help from the femme fatale 'Player' in order to mislead the Detective. If this all sounds eerily familiar, you would not be far wrong. The film was essentially remade by Nicolas Winding Refn as Drive (2011) and in Edgar Wright's 2017 Baby Driver. Yet The Driver offers something that these neon-noir variants do not. In particular, the car chases around Los Angeles are some of the most captivating I've seen: they aren't thrilling in the sense of tyre squeals, explosions and flying boxes, but rather the vehicles come across like wild animals hunting one another. This feels especially so when the police are hunting The Driver, which feels less like a low-stakes game of cat and mouse than a pack of feral animals working together a gang who will tear apart their prey if they find him. In contrast to the undercar neon glow of the Fast & Furious franchise, the urban realism backdrop of the The Driver's LA metropolis contributes to a sincere feeling of artistic fidelity as well. To be sure, most of this is present in the truly-excellent Drive, where the chase scenes do really communicate a credible sense of stakes. But the substitution of The Driver's grit with Drive's soft neon tilts it slightly towards that common affliction of crime movies: style over substance. Nevertheless, I can highly recommend watching The Driver and Drive together, as it can tell you a lot about the disconnected socioeconomic practices of the 1980s compared to the 2010s. More than that, however, the pseudo-1980s synthwave soundtrack of Drive captures something crucial to analysing the world of today. In particular, these 'sounds from the past filtered through the present' bring to mind the increasing role of nostalgia for lost futures in the culture of today, where temporality and pop culture references are almost-exclusively citational and commemorational.
The Souvenir (2019) The ostensible outline of this quietly understated film follows a shy but ambitious film student who falls into an emotionally fraught relationship with a charismatic but untrustworthy older man. But that doesn't quite cover the plot at all, for not only is The Souvenir a film about a young artist who is inspired, derailed and ultimately strengthened by a toxic relationship, it is also partly a coming-of-age drama, a subtle portrait of class and, finally, a film about the making of a film. Still, one of the geniuses of this truly heartbreaking movie is that none of these many elements crowds out the other. It never, ever feels rushed. Indeed, there are many scenes where the camera simply 'sits there' and quietly observes what is going on. Other films might smother themselves through references to 18th-century oil paintings, but The Souvenir somehow evades this too. And there's a certain ring of credibility to the story as well, no doubt in part due to the fact it is based on director Joanna Hogg's own experiences at film school. A beautifully observed and multi-layered film; I'll be happy if the sequel is one-half as good.
The Wrestler (2008) Randy 'The Ram' Robinson is long past his prime, but he is still rarin' to go in the local pro-wrestling circuit. Yet after a brutal beating that seriously threatens his health, Randy hangs up his tights and pursues a serious relationship... and even tries to reconnect with his estranged daughter. But Randy can't resist the lure of the ring, and readies himself for a comeback. The stage is thus set for Darren Aronofsky's The Wrestler, which is essentially about what drives Randy back to the ring. To be sure, Randy derives much of his money from wrestling as well as his 'fitness', self-image, self-esteem and self-worth. Oh, it's no use insisting that wrestling is fake, for the sport is, needless to say, Randy's identity; it's not for nothing that this film is called The Wrestler. In a number of ways, The Sound of Metal (2019) is both a reaction to (and a quiet remake of) The Wrestler, if only because both movies utilise 'cool' professions to explore such questions of identity. But perhaps simply when The Wrestler was produced makes it the superior film. Indeed, the role of time feels very important for the Wrestler. In the first instance, time is clearly taking its toll on Randy's body, but I felt it more strongly in the sense this was very much a pre-2008 film, released on the cliff-edge of the global financial crisis, and the concomitant precarity of the 2010s. Indeed, it is curious to consider that you couldn't make The Wrestler today, although not because the relationship to work has changed in any fundamentalway. (Indeed, isn't it somewhat depressing the realise that, since the start of the pandemic and the 'work from home' trend to one side, we now require even more people to wreck their bodies and mental health to cover their bills?) No, what I mean to say here is that, post-2016, you cannot portray wrestling on-screen without, how can I put it, unwelcome connotations. All of which then reminds me of Minari's notorious red hat... But I digress. The Wrestler is a grittily stark darkly humorous look into the life of a desperate man and a sorrowful world, all through one tragic profession.
Thief (1981) Frank is an expert professional safecracker and specialises in high-profile diamond heists. He plans to use his ill-gotten gains to retire from crime and build a life for himself with a wife and kids, so he signs on with a top gangster for one last big score. This, of course, could be the plot to any number of heist movies, but Thief does something different. Similar to The Wrestler and The Driver (see above) and a number of other films that I watched this year, Thief seems to be saying about our relationship to work and family in modernity and postmodernity. Indeed, the 'heist film', we are told, is an understudied genre, but part of the pleasure of watching these films is said to arise from how they portray our desired relationship to work. In particular, Frank's desire to pull off that last big job feels less about the money it would bring him, but a displacement from (or proxy for) fulfilling some deep-down desire to have a family or indeed any relationship at all. Because in theory, of course, Frank could enter into a fulfilling long-term relationship right away, without stealing millions of dollars in diamonds... but that's kinda the entire point: Frank needing just one more theft is an excuse to not pursue a relationship and put it off indefinitely in favour of 'work'. (And being Federal crimes, it also means Frank cannot put down meaningful roots in a community.) All this is communicated extremely subtly in the justly-lauded lowkey diner scene, by far the best scene in the movie. The visual aesthetic of Thief is as if you set The Warriors (1979) in a similarly-filthy Chicago, with the Xenophon-inspired plot of The Warriors replaced with an almost deliberate lack of plot development... and the allure of The Warriors' fantastical criminal gangs (with their alluringly well-defined social identities) substituted by a bunch of amoral individuals with no solidarity beyond the immediate moment. A tale of our time, perhaps. I should warn you that the ending of Thief is famously weak, but this is a gritty, intelligent and strangely credible heist movie before you get there.
Uncut Gems (2019) The most exhausting film I've seen in years; the cinematic equivalent of four cups of double espresso, I didn't even bother even trying to sleep after downing Uncut Gems late one night. Directed by the two Safdie Brothers, it often felt like I was watching two films that had been made at the same time. (Or do I mean two films at 2X speed?) No, whatever clumsy metaphor you choose to adopt, the unavoidable effect of this film's finely-tuned chaos is an uncompromising and anxiety-inducing piece of cinema. The plot follows Howard as a man lost to his countless vices mostly gambling with a significant side hustle in adultery, but you get the distinct impression he would be happy with anything that will give him another high. A true junkie's junkie, you might say. You know right from the beginning it's going to end in some kind of disaster, the only question remaining is precisely how and what. Portrayed by an (almost unrecognisable) Adam Sandler, there's an uncanny sense of distance in the emotional chasm between 'Sandler-as-junkie' and 'Sandler-as-regular-star-of-goofy-comedies'. Yet instead of being distracting and reducing the film's affect, this possibly-deliberate intertextuality somehow adds to the masterfully-controlled mayhem. My heart races just at the memory. Oof.
Woman in the Dunes (1964) I ended up watching three films that feature sand this year: Denis Villeneuve's Dune (2021), Lawrence of Arabia (1962) and Woman in the Dunes. But it is this last 1964 film by Hiroshi Teshigahara that will stick in my mind in the years to come. Sure, there is none of the Medician intrigue of Dune or the Super Panavision-70 of Lawrence of Arabia (or its quasi-orientalist score, itself likely stolen from Anton Bruckner's 6th Symphony), but Woman in the Dunes doesn't have to assert its confidence so boldly, and it reveals the enormity of its plot slowly and deliberately instead. Woman in the Dunes never rushes to get to the film's central dilemma, and it uncovers its terror in little hints and insights, all whilst establishing the daily rhythm of life. Woman in the Dunes has something of the uncanny horror as Dogtooth (see above), as well as its broad range of potential interpretations. Both films permit a wide array of readings, without resorting to being deliberately obscurantist or being just plain random it is perhaps this reason why I enjoyed them so much. It is true that asking 'So what does the sand mean?' sounds tediously sophomoric shorn of any context, but it somehow applies to this thoughtfully self-contained piece of cinema.
A Quiet Place (2018) Although A Quiet Place was not actually one of the best films I saw this year, I'm including it here as it is certainly one of the better 'mainstream' Hollywood franchises I came across. Not only is the film very ably constructed and engages on a visceral level, I should point out that it is rare that I can empathise with the peril of conventional horror movies (and perhaps prefer to focus on its cultural and political aesthetics), but I did here. The conceit of this particular post-apocalyptic world is that a family is forced to live in almost complete silence while hiding from creatures that hunt by sound alone. Still, A Quiet Place engages on an intellectual level too, and this probably works in tandem with the pure 'horrorific' elements and make it stick into your mind. In particular, and to my mind at least, A Quiet Place a deeply American conservative film below the surface: it exalts the family structure and a certain kind of sacrifice for your family. (The music often had a passacaglia-like strain too, forming a tombeau for America.) Moreover, you survive in this dystopia by staying quiet that is to say, by staying stoic suggesting that in the wake of any conflict that might beset the world, the best thing to do is to keep quiet. Even communicating with your loved ones can be deadly to both of you, so not emote, acquiesce quietly to your fate, and don't, whatever you do, speak up. (Or join a union.) I could go on, but The Quiet Place is more than this. It's taut and brief, and despite cinema being an increasingly visual medium, it encourages its audience to develop a new relationship with sound.
Heart of Darkness (1899) Joseph Conrad Heart of Darkness tells the story of Charles Marlow, a sailor who accepts an assignment from a Belgian trading company as a ferry-boat captain in the African interior, and the novella is widely regarded as a critique of European colonial rule in Africa. Loosely remade by Francis Ford Coppola as Apocalypse Now (1979), I started this book with the distinct possibility that this superb film adaptation would, for a rare treat, be 'better than the book'. However, Conrad demolished this idea of mine within two chapters, yet also elevated the film to a new level as well. This was chiefly due to how observant Conrad was of the universals that make up human nature. Some of his insight pertains to the barbarism of the colonialists, of course, but Conrad applies his shrewd acuity to the at the smaller level as well. Some of these quotes are justly famous: Ah! but it was something to have at least a choice of nightmares, for example, as well as the reference to a fastidiously turned-out colonial administrator who, with unimaginable horrors occurring mere yards from his tent, we learn he was devoted to his books, which were in applepie order . (It seems to me to be deliberately unclear whether his devotion arises from gross inhumanity, utter denial or some combination of the two.) Oh, and there's a favourite moment of mine when a character remarks that It was very fine for a time, but after a bit I did get tired of resting. Tired of resting! Yes, it's difficult to now say something original about a many-layered classic such as this, especially one that has analysed from so many angles already; from a literary perspective at first, of course, but much later from a critical postcolonial perspective, such as in Chinua Achebe's noted 1975 lecture, An Image of Africa. Indeed, the history of criticism in the twentieth century of Heart of Darkness must surely parallel the social and political developments in the Western world. (On a highly related note, the much-cited non-fiction book King Leopold's Ghost is on my reading list for 2022.) I will therefore limit myself to saying that the boat physically falling apart as it journeys deeper into the Congo may be intended to represent that our idea of 'Western civilisation' ceases to function, both morally as well as physically, in this remote environment. And, whilst I'm probably not the first to notice the potential ambiguity, when Marlow lies to Kurtz's 'Intended [wife]' in the closing section in order to save her from being exposed to the truth about Kurtz (surely a metaphor about the ignorance of the West whilst also possibly incorporating some comment on gender?), the Intended replies: I knew it. For me, though, it is not beyond doubt that what the Intended 'knows' is that she knew that Marlow would lie to her: in other words, that the alleged ignorance of everyday folk in the colonial homeland is studied and deliberate. Compact and fairly easy-to-read, it is clear that Heart of Darkness rewards even the most rudimentary analysis.
Rebecca (1938) Daphne du Maurier Daphne du Maurier creates in Rebecca a credible and suffocating atmosphere in the shape of Manderley, a grand English mansion owned by aristocratic widower Maxim de Winter. Our unnamed narrator (a young woman seemingly na ve in the ways of the world) meets Max in Monte Carlo, and she soon becomes the second Mrs. de Winter. The tale takes a turn to the 'gothic', though, when it becomes apparent that the unemotional Max, as well as potentially Manderley itself, appears to be haunted by the memory of his late first wife, the titular Rebecca. Still, Rebecca is less of a story about supernatural ghosts than one about the things that can haunt our minds. For Max, this might be something around guilt; for our narrator, the class-centered fear that she will never fit in. Besides, Rebecca doesn't need an actual ghost when you have Manderley's overbearing housekeeper, Mrs Danvers, surely one of the creepiest characters in all of fiction. Either way, the conflict of a kind between the fears of the protagonists means that they never really connect with each other. The most obvious criticism of Rebecca is that the main character is unreasonably weak and cannot quite think or function on her own. (Isn't it curious that the trait of the male 'everyman' is a kind of physical clumsiness yet the female equivalent is shorthanded by being slightly slow?) But the na vete of Rebecca's narrator makes her easier to relate to in a way, and it also makes the reader far more capable of empathising with her embarrassment. This is demonstrated best whilst she, in one of the best evocations of this particular anxiety I have yet come across, is gingerly creeping around Manderlay and trying to avoid running into the butler. A surprise of sorts comes in the latter stages of the book, and this particular twist brings us into contact with a female character who is anything but 'credulous'. This revelation might even change your idea of who the main character of this book really is too. (Speaking of amateur literary criticism, I have many fan theories about Rebecca, including that Maxim de Winter's estate manager, Frank Crawley, is actually having an affair with Max, and also that Maxim may have a lot more involvement in Mrs Danvers final act that he lets on.) An easily accessible novel (with a great-but-not-perfect 1940 adaptation by Alfred Hitchcock, Rebecca is a real indulgence.
A Clockwork Orange (1962) Anthony Burgess One of Stanley Kubrick's most prominent tricks was to use different visual languages in order to prevent the audience from immediately grasping the underlying story. In his 1975 Barry Lyndon, for instance, the intentionally sluggish pacing and elusive characters require significant digestion to fathom and appreciate, and the luminous and quasi-Renaissance splendour of the cinematography does its part to constantly distract the viewer from the film's greater meaning. This is very much the case in Kubrick's A Clockwork Orange as well whilst it ostensibly appears to be about a Saturnalia of violence, the 'greater meaning' of A Clockwork Orange pertains to the Christian conception of free will; admittedly, a much drier idea to bother making a film around. This is all made much clearer when reading Anthony Burgess' 1962 original novel. Alex became a 'true Christian' through the experimental rehabilitation process, and even offers to literally turn the other cheek at one point. But as Alex had no choice to do so (and can no longer choose to commit violence), he is incapable of making a free moral choice. Thus, is he really a Man? Yet whilst the book's central concern is our conception of free will in modern societies, it also appears to be a repudiation of two conservative principles. Firstly, A Clockwork Orange demolishes the idea that 'high art' leads to morally virtuous citizens. After all, if you can do a bit of the old ultra-violence whilst listening to the glorious 9th by old Ludvig van, then so much for the oft-repeated claims that culture makes you better as a person. (This, at least, I already knew from personal experience.) The other repudiation in A Clockwork Orange is in regard to the pervasive idea that the countryside is a refuge from crime and sin. By contrast, we see the gang commit their most horrific violence in rural areas, and, later, Alex is taken to the countryside by his former droogs for a savage beating. Although this doesn't seem to quite fit the novel, this was actually an important point for Burgess to include: otherwise his book could easily be read as a commentary on the corrupting influence of urban spaces, rather than of modernity itself. The language of this book cannot escape comment here. Alex narrates most of the book in a language called Nadsat, a fractured slang constructed by Burgess based on Russian and Cockney rhyming slang. (The language is strange for only a few pages, I promise. And note that 'Alex' is a very common Russian name.) Using Nadsat has the effect of making the book feel distinctly alien, but it also prevents it from prematurely aging too. Indeed, it comes as bit of a shock to realise that A Clockwork Orange was published 1962, the same year as The Beatles' released their first single, Love Me Do. I could probably say a whole lot more about this thoroughly engrossing book and its movie adaptation (eg. the meta-textual line in Kubrick's version: It's funny how the colours of the real world only seem really real when you watch them on a screen... appears verbatim in the textual original), but I'll leave it there. The book of A Clockwork Orange is not only worth the investment in the language, but is, again, somehow better than the film.
The Great Gatsby (1925) F. Scott Fitzgerald I'm actually being a little deceitful by including this book here: I cannot really say that The Great Gatsby was a 'favourite' read of the year, but its literary merit is so undeniable (and my respect for Fitzgerald's achievement is deep enough) that the experience was one of those pleasures you feel at seeing anything done well. Here you have a book so rich in symbolic meaning that you could easily confuse the experience with drinking Coke syrup undiluted. And a text that has made the difficulty and complexity of reading character a prominent theme of the novel, as well as a technical concern of the book itself. Yet at all times you have in your mind that The Great Gatsby is first and foremost a book about a man writing a book, and, therefore, about the construction of stories and myths. What is the myth being constructed in Gatsby? The usual answer today is that the book is really about the moral virtues of America. Or, rather, the lack thereof. Indeed, as James Boice wrote in 2016:
Could Wilson have killed Gatsby any other way? Could he have ran him over, or poisoned him, or attacked him with a knife? Not at all this an American story, the quintessential one, so Gatsby could have only died the quintessential American death.The quintessential American death is, of course, being killed with a gun. Whatever your own analysis, The Great Gatsby is not only magnificently written, but it is captivating to the point where references intrude many months later. For instance, when reading something about Disney's 'princess culture', I was reminded of when Daisy says of her daughter: I hope she'll be a fool that's the best thing of a girl can be in this world, a beautiful little fool . Or the billboard with the eyes of 'Doctor T. J. Eckleburg'. Or the fact that the books in Gatsby's library have never been read (so what is 'Owl Eyes' doing there during the party?!). And the only plain room in Gatsby's great house is his bedroom... Okay, fine, I must have been deluding myself: I love this novel.
A Wizard of Earthsea (1971) Ursula K. Le Guin How did it come to be that Harry Potter is the publishing sensation of the century, yet Ursula K. Le Guin's Earthsea is only a popular cult novel? Indeed, the comparisons and unintentional intertextuality with Harry Potter are entirely unavoidable when reading this book, and, in almost every respect, Ursula K. Le Guin's universe comes out the victor. In particular, the wizarding world that Le Guin portrays feels a lot more generous and humble than the class-ridden world of Hogwarts School of Witchcraft and Wizardry. Just to take one example from many, in Earthsea, magic turns out to be nurtured in a bottom-up manner within small village communities, in almost complete contrast to J. K. Rowling's concept of benevolent government departments and NGOs-like institutions, which now seems a far too New Labour for me. Indeed, imagine an entire world imbued with the kindly benevolence of Dumbledore, and you've got some of the moral palette of Earthsea. The gently moralising tone that runs through A Wizard of Earthsea may put some people off:
Vetch had been three years at the School and soon would be made Sorcerer; he thought no more of performing the lesser arts of magic than a bird thinks of flying. Yet a greater, unlearned skill he possessed, which was the art of kindness.Still, these parables aimed directly at the reader are fairly rare, and, for me, remain on the right side of being mawkish or hectoring. I'm thus looking forward to reading the next two books in the series soon.
Blood Meridian (1985) Cormac McCarthy Blood Meridian follows a band of American bounty hunters who are roaming the Mexican-American borderlands in the late 1840s. Far from being remotely swashbuckling, though, the group are collecting scalps for money and killing anyone who crosses their path. It is the most unsparing treatment of American genocide and moral depravity I have ever come across, an anti-Western that flouts every convention of the genre. Blood Meridian thus has a family resemblance to that other great anti-Western, Once Upon a Time in the West: after making a number of gun-toting films that venerate the American West (ie. his Dollars Trilogy), Sergio Leone turned his cynical eye to the western. Yet my previous paragraph actually euphemises just how violent Blood Meridian is. Indeed, I would need to be a much better writer (indeed, perhaps McCarthy himself) to adequately 0utline the tone of this book. In a certain sense, it's less than you read this book in a conventional sense, but rather that you are forced to witness successive chapters of grotesque violence... all occurring for no obvious reason. It is often said that books 'subvert' a genre and, indeed, I implied as such above. But the term subvert implies a kind of Puck-like mischievousness, or brings to mind court jesters licensed to poke fun at the courtiers. By contrast, however, Blood Meridian isn't funny in the slightest. There isn't animal cruelty per se, but rather wanton negligence of another kind entirely. In fact, recalling a particular passage involving an injured horse makes me feel physically ill. McCarthy's prose is at once both baroque in its language and thrifty in its presentation. As Philip Connors wrote back in 2007, McCarthy has spent forty years writing as if he were trying to expand the Old Testament, and learning that McCarthy grew up around the Church therefore came as no real surprise. As an example of his textual frugality, I often looked for greater precision in the text, finding myself asking whether who a particular 'he' is, or to which side of a fight some two men belonged to. Yet we must always remember that there is no precision to found in a gunfight, so this infidelity is turned into a virtue. It's not that these are fair fights anyway, or even 'murder': Blood Meridian is just slaughter; pure butchery. Murder is a gross understatement for what this book is, and at many points we are grateful that McCarthy spares us precision. At others, however, we can be thankful for his exactitude. There is no ambiguity regarding the morality of the puppy-drowning Judge, for example: a Colonel Kurtz who has been given free license over the entire American south. There is, thank God, no danger of Hollywood mythologising him into a badass hero. Indeed, we must all be thankful that it is impossible to film this ultra-violent book... Indeed, the broader idea of 'adapting' anything to this world is, beyond sick. An absolutely brutal read; I cannot recommend it highly enough.
Bodies of Light (2014) Sarah Moss Bodies of Light is a 2014 book by Glasgow-born Sarah Moss on the stirrings of women's suffrage within an arty clique in nineteenth-century England. Set in the intellectually smoggy cities of Manchester and London, this poignant book follows the studiously intelligent Alethia 'Ally' Moberly who is struggling to gain the acceptance of herself, her mother and the General Medical Council. You can read my full review from July.
House of Leaves (2000) Mark Z. Danielewski House of Leaves is a remarkably difficult book to explain. Although the plot refers to a fictional documentary about a family whose house is somehow larger on the inside than the outside, this quotidian horror premise doesn't explain the complex meta-commentary that Danielewski adds on top. For instance, the book contains a large number of pseudo-academic footnotes (many of which contain footnotes themselves), with references to scholarly papers, books, films and other articles. Most of these references are obviously fictional, but it's the kind of book where the joke is that some of them are not. The format, structure and typography of the book is highly unconventional too, with extremely unusual page layouts and styles. It's the sort of book and idea that should be a tired gimmick but somehow isn't. This is particularly so when you realise it seems specifically designed to create a fandom around it and to manufacturer its own 'cult' status, something that should be extremely tedious. But not only does this not happen, House of Leaves seems to have survived through two exhausting decades of found footage: The Blair Witch Project and Paranormal Activity are, to an admittedly lesser degree, doing much of the same thing as House of Leaves. House of Leaves might have its origins in Nabokov's Pale Fire or even Derrida's Glas, but it seems to have more in common with the claustrophobic horror of Cube (1997). And like all of these works, House of Leaves book has an extremely strange effect on the reader or viewer, something quite unlike reading a conventional book. It wasn't so much what I got out of the book itself, but how it added a glow to everything else I read, watched or saw at the time. An experience.
Milkman (2018) Anna Burns This quietly dazzling novel from Irish author Anna Burns is full of intellectual whimsy and oddball incident. Incongruously set in 1970s Belfast during The Irish Troubles, Milkman's 18-year-old narrator (known only as middle sister ), is the kind of dreamer who walks down the street with a Victorian-era novel in her hand. It's usually an error for a book that specifically mention other books, if only because inviting comparisons to great novels is grossly ill-advised. But it is a credit to Burns' writing that the references here actually add to the text and don't feel like they are a kind of literary paint by numbers. Our humble narrator has a boyfriend of sorts, but the figure who looms the largest in her life is a creepy milkman an older, married man who's deeply integrated in the paramilitary tribalism. And when gossip about the narrator and the milkman surfaces, the milkman beings to invade her life to a suffocating degree. Yet this milkman is not even a milkman at all. Indeed, it's precisely this kind of oblique irony that runs through this daring but darkly compelling book.
The First Fifteen Lives of Harry August (2014) Claire North Harry August is born, lives a relatively unremarkable life and finally dies a relatively unremarkable death. Not worth writing a novel about, I suppose. But then Harry finds himself born again in the very same circumstances, and as he grows from infancy into childhood again, he starts to remember his previous lives. This loop naturally drives Harry insane at first, but after finding that suicide doesn't stop the quasi-reincarnation, he becomes somewhat acclimatised to his fate. He prospers much better at school the next time around and is ultimately able to make better decisions about his life, especially when he just happens to know how to stay out of trouble during the Second World War. Yet what caught my attention in this 'soft' sci-fi book was not necessarily the book's core idea but rather the way its connotations were so intelligently thought through. Just like in a musical theme and varations, the success of any concept-driven book is far more a product of how the implications of the key idea are played out than how clever the central idea was to begin with. Otherwise, you just have another neat Borges short story: satisfying, to be sure, but in a narrower way. From her relatively simple premise, for example, North has divined that if there was a community of people who could remember their past lives, this would actually allow messages and knowledge to be passed backwards and forwards in time. Ah, of course! Indeed, this very mechanism drives the plot: news comes back from the future that the progress of history is being interfered with, and, because of this, the end of the world is slowly coming. Through the lives that follow, Harry sets out to find out who is passing on technology before its time, and work out how to stop them. With its gently-moralising romp through the salient historical touchpoints of the twentieth century, I sometimes got a whiff of Forrest Gump. But it must be stressed that this book is far less certain of its 'right-on' liberal credentials than Robert Zemeckis' badly-aged film. And whilst we're on the topic of other media, if you liked the underlying conceit behind Stuart Turton's The Seven Deaths of Evelyn Hardcastle yet didn't enjoy the 'variations' of that particular tale, then I'd definitely give The First Fifteen Lives a try. At the very least, 15 is bigger than 7. More seriously, though, The First Fifteen Lives appears to reflect anxieties about technology, particularly around modern technological accelerationism. At no point does it seriously suggest that if we could somehow possess the technology from a decade in the future then our lives would be improved in any meaningful way. Indeed, precisely the opposite is invariably implied. To me, at least, homo sapiens often seems to be merely marking time until we can blow each other up and destroying the climate whilst sleepwalking into some crisis that might precipitate a thermonuclear genocide sometimes seems to be built into our DNA. In an era of cli-fi fiction and our non-fiction newspaper headlines, to label North's insight as 'prescience' might perhaps be overstating it, but perhaps that is the point: this destructive and negative streak is universal to all periods of our violent, insecure species.
The Goldfinch (2013) Donna Tartt After Breaking Bad, the second biggest runaway success of 2014 was probably Donna Tartt's doorstop of a novel, The Goldfinch. Yet upon its release and popular reception, it got a significant number of bad reviews in the literary press with, of course, an equal number of predictable think pieces claiming this was sour grapes on the part of the cognoscenti. Ah, to be in 2014 again, when our arguments were so much more trivial. For the uninitiated, The Goldfinch is a sprawling bildungsroman that centres on Theo Decker, a 13-year-old whose world is turned upside down when a terrorist bomb goes off whilst visiting the Metropolitan Museum of Art, killing his mother among other bystanders. Perhaps more importantly, he makes off with a painting in order to fulfil a promise to a dying old man: Carel Fabritius' 1654 masterpiece The Goldfinch. For the next 14 years (and almost 800 pages), the painting becomes the only connection to his lost mother as he's flung, almost entirely rudderless, around the Western world, encountering an array of eccentric characters. Whatever the critics claimed, Tartt's near-perfect evocation of scenes, from the everyday to the unimaginable, is difficult to summarise. I wouldn't label it 'cinematic' due to her evocation of the interiority of the characters. Take, for example: Even the suggestion that my father had close friends conveyed a misunderstanding of his personality that I didn't know how to respond it's precisely this kind of relatable inner subjectivity that cannot be easily conveyed by film, likely is one of the main reasons why the 2019 film adaptation was such a damp squib. Tartt's writing is definitely not 'impressionistic' either: there are many near-perfect evocations of scenes, even ones we hope we cannot recognise from real life. In particular, some of the drug-taking scenes feel so credibly authentic that I sometimes worried about the author herself. Almost eight months on from first reading this novel, what I remember most was what a joy this was to read. I do worry that it won't stand up to a more critical re-reading (the character named Xandra even sounds like the pharmaceuticals she is taking), but I think I'll always treasure the first days I spent with this often-beautiful novel.
Beyond Black (2005) Hilary Mantel Published about five years before the hyperfamous Wolf Hall (2004), Hilary Mantel's Beyond Black is a deeply disturbing book about spiritualism and the nature of Hell, somewhat incongruously set in modern-day England. Alison Harte is a middle-aged physic medium who works in the various towns of the London orbital motorway. She is accompanied by her stuffy assistant, Colette, and her spirit guide, Morris, who is invisible to everyone but Alison. However, this is no gentle and musk-smelling world of the clairvoyant and mystic, for Alison is plagued by spirits from her past who infiltrate her physical world, becoming stronger and nastier every day. Alison's smiling and rotund persona thus conceals a truly desperate woman: she knows beyond doubt the terrors of the next life, yet must studiously conceal them from her credulous clients. Beyond Black would be worth reading for its dark atmosphere alone, but it offers much more than a chilling and creepy tale. Indeed, it is extraordinarily observant as well as unsettlingly funny about a particular tranche of British middle-class life. Still, the book's unnerving nature that sticks in the mind, and reading it noticeably changed my mood for days afterwards, and not necessarily for the best.
The Wall (2019) John Lanchester The Wall tells the story of a young man called Kavanagh, one of the thousands of Defenders standing guard around a solid fortress that envelopes the British Isles. A national service of sorts, it is Kavanagh's job to stop the so-called Others getting in. Lanchester is frank about what his wall provides to those who stand guard: the Defenders of the Wall are conscripted for two years on the Wall, with no exceptions, giving everyone in society a life plan and a story. But whilst The Wall is ostensibly about a physical wall, it works even better as a story about the walls in our mind. In fact, the book blends together of some of the most important issues of our time: climate change, increasing isolation, Brexit and other widening societal divisions. If you liked P. D. James' The Children of Men you'll undoubtedly recognise much of the same intellectual atmosphere, although the sterility of John Lanchester's dystopia is definitely figurative and textual rather than literal. Despite the final chapters perhaps not living up to the world-building of the opening, The Wall features a taut and engrossing narrative, and it undoubtedly warrants even the most cursory glance at its symbolism. I've yet to read something by Lanchester I haven't enjoyed (even his short essay on cheating in sports, for example) and will be definitely reading more from him in 2022.
The Only Story (2018) Julian Barnes The Only Story is the story of Paul, a 19-year-old boy who falls in love with 42-year-old Susan, a married woman with two daughters who are about Paul's age. The book begins with how Paul meets Susan in happy (albeit complicated) circumstances, but as the story unfolds, the novel becomes significantly more tragic and moving. Whilst the story begins from the first-person perspective, midway through the book it shifts into the second person, and, later, into the third as well. Both of these narrative changes suggested to me an attempt on the part of Paul the narrator (if not Barnes himself), to distance himself emotionally from the events taking place. This effect is a lot more subtle than it sounds, however: far more prominent and devastating is the underlying and deeply moving story about the relationship ends up. Throughout this touching book, Barnes uses his mastery of language and observation to avoid the saccharine and the maudlin, and ends up with a heart-wrenching and emotive narrative. Without a doubt, this is the saddest book I read this year.
Series: | Fractured Fables #1 |
Publisher: | Tordotcom |
Copyright: | 2021 |
ISBN: | 1-250-76536-6 |
Format: | Kindle |
Pages: | 121 |
"You know it wasn't originally a spinning wheel in the story?" I offer, because alcohol transforms me into a chatty Wikipedia page.A Spindle Splintered is told from Zinnia's first-person perspective, and Zinnia is great. My favorite thing about Harrow's writing is the fierce and complex emotions of her characters. The overall tone is lighter than The Once and Future Witches or The Ten Thousand Doors of January, but Harrow doesn't shy away from showing the reader Zinnia's internal thought process about her illness (and her eye-rolling bemusement at some of the earlier emotional stages she went through).
Dying girl rule #3 is no romance, because my entire life is one long trolley problem and I don't want to put any more bodies on the tracks. (I've spent enough time in therapy to know that this isn't "a healthy attitude towards attachment," but I personally feel that accepting my own imminent mortality is enough work without also having a healthy attitude about it.)There's a content warning for parents here, since Harrow spends some time on the reaction of Zinnia's parents and the complicated dance between hope, despair, smothering, and freedom that she and they had to go through. There were no easy answers and all balances were fragile, but Zinnia always finds her feet. For me, Harrow's character writing is like emotional martial arts: rolling with punches, taking falls, clear-eyed about the setbacks, but always finding a new point of stability to punch back at the world. Zinnia adds just enough teenage irreverence and impatience to blunt the hardest emotional hits. I really enjoy reading it. The one caution I will make about that part of the story is that the focus is on Zinnia's projected lifespan and not on her illness specifically. Harrow uses it as setup to dig into how she and her parents would react to that knowledge (and I thought those parts were done well), but it's told from the perspective of "what would you do if you knew your date of death," not from the perspective of someone living with a disability. It is to some extent disability as plot device, and like the fairy tale that it's based on, it's deeply invested in the "find a cure" approach to the problem. I'm not disabled and am not the person to ask about how well a story handles disability, but I suspect this one may leave something to be desired. I thought the opening of this story is great. Zinnia is a great first-person protagonist and the opening few chapters are overflowing with snark and acerbic commentary. Dumping Zinnia into another world but having text messaging still work is genius, and I kind of wish Harrow had made that even more central to the book. The rest of the story was good but not as good, and the ending was somewhat predictable and a bit of a deus ex machina. But the characters carried it throughout, and I will happily read more of this. Recommended, with the caveat about disability and the content warning for parents. Followed by A Mirror Mended, which I have already pre-ordered. Rating: 8 out of 10
Series: | Raybearer #1 |
Publisher: | Amulet Books |
Copyright: | 2020 |
ISBN: | 1-68335-719-1 |
Format: | Kindle |
Pages: | 308 |
For the kid scanning fairy tales for a hero with a face like theirs. And for the girls whose stories we compressed into pities and wonders, triumphs and cautions, without asking, even once, for their names.I think she was successful on both parts of that promise, and it makes for some great reading. Recommended. Followed by Redemptor. Rating: 8 out of 10
Next.